A channel subsystem (CSS) is a conventional part of a data processing system and is well known in the art of data processing systems. A CSS directs the flow of information between I/O devices and main storage and is mainly comprised of one or more I/O processors (IOPs) and I/O channel paths (channels) with some participation of the central processors (CPs). IOPs are also synonymously referred to as SAPs (system assist processors). In a simple system, I/O instructions are initiated by a CP that might execute a sequence of instructions that partly use only the resources of the CP and that partly directly control the resources and the operations of the CSS. In a CSS of the type that will use this invention, IOPs perform a portion of an I/O operation, and one or more channels handle other parts of the I/O operation. The IOPs determine which channel to select for an I/O operation. The channels handle the actual data transfers into and out of processor memory and they execute commands by forming orders that are sent to the I/O device controllers. The IOP also handles general parts of the I/O operation such as communicating with the CPs through interfaces such as control blocks in reserved memory called hardware system area (HSA), scheduling the I/O operations, and reporting status conditions to a CP. A single IOP can handle requests from multiple CPs and report status to different CPs.
The IOPs and the channels also communicate with each other, and the present invention provides an improvement in these communications. The IOP signals a particular channel when an I/O operation is to be performed and the channel signals the IOP when the operation has been completed and whenever some other general operation is to be performed by the IOP. In the known prior art, these communications are carried out over signal wires that run between the IOP and the channels or over the existing data paths that connect the IOP and the channels.
In IBM mainframe systems from the 3090 to the z990, an integral part of the communication mechanism used in the Channel Subsystem (CSS) between the I/O Processors (IOPs) and the Channels for signaling and passing messages to each other is a hardware array called the Channel Communication Array (CCA) as illustrated in FIG. 1. Messages directed to the channel (TO_CHN) typically contain commands to instruct the Channel to perform the necessary steps to initiate I/O operations, such as Start Subchannels (SSCH) to I/O devices. Messages directed to an IOP typically contain the status of these I/O operations as well as unsolicited status from devices connected to the channel. The SSCH instruction and the resulting status is described in the zSeries Principle of Operations, IBM Corporation, May 2004, Fourth Edition, SA22-7832-03.
There is one eight byte CCA slot (“CCA”) for each prior art Channel as shown in FIG. 1. It is bidirectional, but half-duplex in nature. It can only contain a message from either the Channel directed to the IOP (TO_IOP CCA) or from the IOP directed to the Channel (TO_CHN CCA). There may be hundreds or thousands of channels in a large mainframe system. The mechanism provides a set of command primitives the IOPs and Channels use to write, read and reset the CCA. To provide serialization and to avoid overwriting a CCA message that has not been processed yet, the mechanism provides for a “compare and swap like” form of write CCA called a “Conditional Write CCA”. When the CMD/STATUS byte of the CCA is nonzero, a “Conditional Write CCA” will return a “CCA Busy” response to the sender who attempted to issue the “Conditional Write CCA”. If there is an unprocessed TO_IOP or TO_CHN message sitting in the CCA, this Conditional Write primitive returns a “CCA busy” response to the sender rather than allowing the new message to overwrite the unprocessed message in the CCA. To read this message and free up the CCA, the sender who received the “CCA Busy” message will issue a “Read and Reset CCA”. The Reset portion of the command primitive will zero out the CMD/STATUS byte thereby giving the sender another opportunity to reissue the “Conditional Write CCA”.
There is also a signaling mechanism to give initiative to the receiver of the CCA message, when it's successfully sent with the “Conditional Write CCA” command, to process the contents of this CCA message. The IOP, as a sender of a CCA message, can signal a specified channel to process the CCA via the Tap Channel primitive. And a Channel, as a sender, can signal an unspecified IOP to process the CCA via setting that channels interrupt vector (IV) bit using the Set IV primitive.
Table A shows a typical format of a TO_CHN or TO_IOP CCA Message 8 byte message:
TABLE AFormat of the 8 byte CCA Message
As an example, when the IOP needs to process a SSCH instruction that was issued by a program running on a CP, as part of that processing, the IOP will need to send a CCA message to a selected channel to process the SSCH for the subchannel (SCB) specified by the program. Note that an SCB is a control block in HSA that contains the controls for managing an I/O operation for a specific device.
The IOP would set up a TO_CHN message as follows: The D bit (Direction Bit) would be set to ‘0’ indicating that this message was a TO_CHN message. The CCA Command Code would be set to ‘01’ to indicate to the channel that this is a “SSCH Initiative CCA Message”. And the Subchannel Identifier would contain addressing information pointing to the specific SCB in HSA describing the device that this SSCH is targeted to. The IOP would then issue a “Conditional Write CCA and Tap Channel” to send the TO_CHN message.
Continuing with this example, the channel hardware would interrupt the channel as the result of the Tap Channel which would cause the channel to issue a Read and Reset CCA. Upon receipt of “SSCH Initiative CCA Message”, the channel would verify that this message is a TO_CHN message by reading the Direction bit in the CCA Message. It would then examine the CCA Command Code in the message and recognize it to be a “SSCH Initiative CCA Message”. It would then process the SSCH. Upon completion of the SSCH, device status will need to be presented to the program.
Upon receipt of ending device status such as channel end, the channel will present this status to the IOP in the SCB. To let the IOP know that it has ending status to present and in which subchannel the status is, the channel will build a TO_CHN message containing: the D bit set to 1 indicating a TO_IOP message, the CCA Command Code set to ‘80’x to indicate “SCB Work Pending message”, and the Subchannel Identifier will contain addressing information pointing to the specific SCB in HSA containing the ending device status. The channel would then issue a “Conditional Write CCA and Set IV bit to send the TO_IOP message and signal the IOP.
Continuing with this example, the IOP hardware would interrupt the IOP as the result of the Set IV which would cause the IOP to issue a Read and Reset CCA. Upon receipt of “SCB Work Pending message”, the IOP would verify that this message is a TO_IOP message by reading the Direction bit in the CCA Message. It would then examine the CCA Command Code in the message and recognize it to be a “SCB Work Pending message”. It would then examine the SCB in HSA addressed by the Subchannel Identifier and present the status of the SSCH to the CP.
At system initialization time, each channel is assigned to one of several IOPs. It is this IOP that will service that channel's CCA and IV bit as well as be the only IOP to write the CCAs and Tap its channel. The channel is said to have “affinity” to that IOP. Note that an IOP can service multiple channels.
As the IOPs and channels have become faster and faster and more efficient at managing multiple concurrent operations, the single message, half-duplex nature of the CCA has become a bottleneck in achieving the desired I/O rates For a FICON channel, it is anticipated that the CCA usage could possibly be 10 to 40 times the rate as it is for an ESCON channel. (FICON stands for Fibre Connection Channel which is an IBM adaptation of the Fibre Channel Standard. ESCON stands for Enterprise System Connection Channel, and is otherwise known as a “serial” channel.) The ratio of CCA Busy responses to the actually successful CCA writes is increasing to the point of diminishing returns. To add to the performance degradation, as mainframes become bigger and more nodal in design—whereby groups of CPs, IOPs and CCA arrays are located on different physical “books”, the path length from IOPs on one book to the CCAs on another book has increased thereby increasing access time to the CCA.
With increased market pressures to use more common, industry standard hardware, the motivation to incrementally improve the existing CCA hardware has waned. Having a solution that uses more industry standard hardware increases the connectivity options available. With these goals and objectives, a new and innovative solution that makes use of commonly available hardware is needed.